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The Oxnard (California) Comounity Mental Health 
Center reports on evaluation of efforts to train parents in ohild 
management skills using behavior modification technicues. Rigorous 
training procedures, curriculum, and ovalua ti*olhi techniques hav€ been 
developed over the pas^ two years. Twenty groups of 3*-15 parents have 
received training in behavior modification during 10 sessions led by 
mental health technicians. The paper emphasizes the iBpor;tance of 
evaluating program outcome in terms of multilevel source^ of data if 
the workshop format is to continue as a v labile approach to child 
therapy. Accordingly^ evaluation pf program progress enc.o^ipass^s the 
traditionally defined process variabl.es such as attendanQe, 
participation by parents in v.orkshops sessions, and characteristics 
of leadership behaviors of the trainers* In jaddition, the evaluation 
of parent training as outlined in this papet is in terms of what 
parents do, rather than what they tal)c abou^^oing. Therefore/ the 
effects of the training program are evaluated in workshop 
role-playing sessions, and later by home and phone contacts. Finally, 
the paper suggests that the impact of par ent workshops should be 
assessed in terms of t^^ reduction of " del inquency rates, admissions 
to residential treatment facrlitieis for children, and dj;opout rates. 
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California. 



The authors wish to express their appreciation for the excellent support given this work 
by Rafael Canton^ M*D, and Stephen Coray, M.D, , Directors of the Ventura County Mental 
Health Department and Health Services Agency respectively.^.- The authors are also indebted 
to the dedicated and enthusiastic group leadership of Edwin Bryan » III » Albert Rivera » 
Gina Manchester » Vlkki Smith, Monica tfyron, Ramon Rocha, Walter Lunsfbrd, Jack Schaff, 
and Lawrence Weathers who have shared the responsibility for training parents in child 
management. 

■3 y . ■ ■ ■ ■ ■ ' 

Project Director, Community-Based Preventive Services for Children and Adolescents and 
Chief of Chlldreofc Services at the Oxnard Mental Health Center. 



Deputy Program Leafier, Ventura County Mentay Health Department and Associate Professor 
of Psychiatry, UCL* S^)^ool of Medicine. ^ 



The training of parAprofcsslo^alo, Indigenous x\OA\i}ro£o3oio:Mo\ end uatural thevc- 
pista (patents » sibl^ga^ achoolmatea) has become a major focus of (Contemporary ^ clinical 
psychology. Propelled by the strong ideological currents of brief therapy, c^'4t*-effectiye« 

hessi and cotomunity mental healthy these training' efforts have been tkt\^ed more by ^nthusf- 

■ * ' ' ■■ . \ \ 

astic description than by sober evaluation. Despite a legacy of eapitricismi behavior modif 

■ ■ ■ , '\ ' 

fiers who have conducted training programs for parents in child management have indulged 
more in optimistic proselytizing than in critical self -^appraisal* (Berkowit^ and Graziano^ 
1972; Arnold, 1973) ' \ 

Evaluation of efforts to train parents in child management skills can produce an 
informative and constructive data base fur advancing behavior technology and prevention * 
of behavioral disorders. Evaluation strategy should proceed from a compiehensiye formula* 
tion of the wide variety of process and outcom^ variables which should be\ measured and 
submitted to experimental and corr.^lative analysis* In Table I is listed \the various 
levels of evaluation for programs training parents in child management/ 

Evaluation of program structure includes the characteristics of the patents (social 
clas3> ethnic group, education, number of children/ marital status, exposure to their own 
parents, attitudes toward trainihg procedures); characteristics of the targeted children 
(age, sex, birth ordar, number of sibs, type of behavioral problem —-typography, fre^^ 
quency duration and reinforcement history of deviant behavior; training and experience 
as leader and parent of "trainers'^; and currlculuin and procedures of the training effort* 
Very few published reports of programs for training parents in ^^hi Id management contain 
specific, operational descriptions of what the training actually consists of, Without 
an adequate and complete description of procedures, replications are impossible and re 
sports of outcome are difficult to interpret or evaluate. 

Some of the dimensions of program structure that should be concretely elucidated 
- tr* any evaluation are: ■■loS: : ■ ' ' " - ■ ■ ' -,^-^:^i;v^^:;::^:■^:^^^^ 

viilrllS;! and screening of parents and childrer^;/!!;!;^ 

set up by Instructipnt attd enthusiasm with which thay are given. 



. 3* Site of training (clinic i conminity pUce. school » home otf combination of 
^ $ltes)» / ^ ' 
4« Format and cuttlculLum of training (amount of each, content and quality) 

a, lecture and reading materials 

b, demonstrations (live, video, transparencies, film) 

c, testimonials and teaching assistance by previously trained parents 

d, frequency and number pf sessions, and follow-up or "booster sessions" 

• e, experiential training (role-playing^ modelling, remote control prompt- 

ing with their own or other child) ^ 
5* Contingencies placed on parents V refundable deposits; fees; fines; feed- 
back contingent upon completion of homework; home visits; and phone, calls), 

6. Instructipn^al objectives* 

7, Number and duration of sessions; nun^ber of parents In group. 

At the Oxnard (Ca,) Community Mental Health Center, where 20 groups of 3-15 parents 
have received training in behavior modification during 10 session workshops led by mental 
health technicians , a highly specific training procedure and curriculum has been developed 
over the past two years (Altchi^on and Llberman, 1973) » A semi-standardized^ procedure for 
trdi{iing group leaders has also evolved. These are first steps in any program evaluation* 

Evaluation of program progress .encompasses the traditionally defined process vairi- 
ables such as attendance, participation by parents in workshop sessions, and characteris- 
tics of the leadership behaviors of the trainers. Progress variables have been carefully 
delineated by behavior modifiers in the *^thods" section of research publications and 
deserve similar attention in evaluations of parent training efforts, f Ft r example, while 
attendance at training sessions might seem like a trivial variable, unless a technology 
Is developed to induce regular attendance, any and all outcome measures are irrelevant, 

A direct > rigorous and reliable means of evaluating workshops could be achlevert by 
aonltorihg the behavior of workshop leaders and participating parents.^ .At the Community 
::|)^||n|||;:J 

det6i!iiii|^^ ' , 

variety of measurement dimensions and procedures are available for evaluating the teaching 



' behr*7tors of group leaders. These Include: (1) non-verbal behavlora sucH as eye contact 
with parents, smiles ) use of hands, use of notes, blackboard, slide projector; (2) number 
of eliciting questions by leader, amount of y^t^^^l- discussion by parents, and the distri*- 
butlcn of parent participation; (3) number of parents who approach leader during breaks 
in session or before and after sessions; <4) number of anecdotal examples given by leader 
to illustrate behavioral principles; and (5) number of interventions, suggested when pro «- 
sentcd with specific problemi. We are currently researching whether these measures dif- 
ferenttiate n«ophyte from experienced group leaders » To the extent that these behaviors 
reflect competence and not solely stylistic or individual differences, we will have iso- 
lated behaviors necessary to accelerate in leaders of parents' workshops. 

But, perhaps the v;idely used lecture format Itself should be changed, Ue have hr;3in 
C':iuck by the apathy and passivi|:y of parent audiences in our workshops. Parents must be 
persistently probed and prompted to respond or they do not interact' at all. They rarely 
ask quostlcns spontaneously. Workshop programs can be designed to maintain the verb?! 
interactions cf parents with the group leaders by incorporating experiential methods such 
as modelling, role^playing or behavior rehearsal. We may discover that parents who have 

"V - 

a history of ^extended television viewing can absdrb and use infomation on behavioral prin 
cipl'^.s if it is carerully and professionally programmed on videotapes. Videotape e ^.n;>le3 
of parer\t-;-hlld interactions provide models for such iJiterventions as, up ing extinction, 
praise or tlT.^-oi:t. The advantages of videotaped behavior training lies not so much i:^ 
the reductici: of professionals' tine but rather in the re-allocation of professionals' 
time to provide imrredi^te reinforcement and feedback to parents as they practice ih nol^- 
playiag the modelled scenes they Jiave Just observed. ; 1^ ^ 

At the Oxnard M':jntal Health Center, attendance at Parent Workshops offered as 

c ftfo, corrjTivnity service has been a helpful measure of Program Progress for cir^rizing 

■ ■ • - ■ f' ' ' 

the Program Structure. IThen the workshops began, attendance for the 10 sessions was unJer 
y^p pftrceat of those comfng tb at least the flr^t session. This dismal reoponrc pronp?:cd 
? us to institute an avoidance contingency with the parents: A $10.00 deposit ia required 

at^ the first session which is returned in full if one or both parents (or a surrogate sent 
E^^^em) attend each and every session. In the 18 workshops held since that contingency 



vas estabilahed, attendance has averaged ovet 75 percent* 

In recent attempts to involve* parents of pre-Jelinqaent juitioir higli tchool stuJontiJ 
in wotkshopQ, ve were unable to achieve the attendance o^ even one parent at the first ' 
session, despite such heroic efforts as calling them on the phonle 2-3' times reminding them 
about the vorHshop^ sending messages about the workshop home with their children, offer- 
ing them transportation to the evening workshop, and in some cases , having a Spanish speak* 
ing Teacher ^8 Aide talk to the parent face*to-face» < Difficulties were also encountered 
in attempts to enlist these parents' cooperation in setting up school /home contingency ^ 
contracts for their children, even when home visits by indigenous nonprofessionals were 
utilised. It is apparent that v/e do not have fin adequate technology to profiK)te the at-- 
tendance and participation of parents from poverty and multi^problem families • 

Evaluation of _ Program Outcome 
I Evaluation of most workshops has focused on changes produced indirectly in chil- 
dren's behavioral principles. Thie changes have been reported by the parents^ with only 
rare substantiation provided by the reliability checks b£ objective observer^* The en- 
thusiasm of the group leaders for their work or the self reports by parents pn their chil- 
dren^s improvement are not adequate for a sclentifiq evaluation of the parent training l[- 
enterprise/ - !. 

A variety of educational formats has been used for parent workshops ih behavior 
modification; however, only rarely has the effectiveness of the instructional procedures 
been examined by direct observation of the parents' behavior* The | workshops! have focused 
on the intellectual training of groups of parents through lectures;, reading jassignnients, 
and verbal discussion of data brought in by the parents. Direct measurement of parental 

behavior has been de-emphaslzcd because of the costs and inconvenience of sending ob^Jer- 

... ^ " • ■ , , _ . N' ■ ' 

vets into the home » and in developing' valid and reliable observational codes. Our major 

. ■ - t ■ . - ■ ■ " " / • - ■ ^ I,^ . ■ ■ „ ■ 

evaluation tools have been the tradltionaf .paper-and-pencfl tests of knowledge obtained 
at>out behavioral principles and-the self^repbrts of parents (with or withoiit graphs) oh 
pbanges they observe in their children* At the Oxnard Mental Health Centeij we have duti- 

■ ■ ■ . ' _ ■ i , ■ ■ 

fully collected test results finding an average percent improvement in prc-post 
Q es and have labored over ^data graphs finding that 62 graphs v/e^c produced by> 



72^parental pairs (or dingles) with 36 succesiiful interventipns and 22 of these grapha 

focur.ing on a secoi»d behavioral Qoal in a child. 

/ i . . • ■ • . ^ ■ ■ 

" In actual! ty> the targeted objectives of parent workshops should be the perfor- 
mance or behavior of parents in altering the behavior of their children. We need be- 
^h^vioral laensuresi ther^, and not teats of conceptual knowledge; We want to know what 
parents do, not. what they talk about doing. The parent workshop model must be recon- 
sidered in terras of^ the real behaviors it will tnodify. 

Role*playing or behavioral rehearsal offer a step in the direction of more re- 
liable measurement of parent£j^ behavior , Wo are developing procedures for evaluating 
sequences of par^int-child interacti'cns performed in role-playing by the parentcf at work- 
shop sessions « If parents cannot perform in rehearsal, then it is not likely that they 
can perform at home; however, the converse is not necessarily true. This approach to 
evaluating outcome of parent workshops offers the following advantages; 

1. performance in role-playing samples the parent's behavioral repertoire and is 
less abstracted from "in home'* pcrfc;:mahce than verbal ^lespcnses on' tests or in 
group discissions. \ , " 

2. '^ role-playing is convenient and Inexpensive^ to observe and rate since parents ^ 
are alrsady In groups at £i single site, \ \ 

3* role -playing may prove to facilitate parental invo^lven^nt in the workshop 

sef?sicn*5, attendance, and carrying out as3ignm^ints in the home. Our preliminary 

evneriencriS with parents of . delinquent boys , ' in a replica of the Achleycmc-nt 

Place model, indicate that the parents particijsate in the workshop to a much 

greater extent when role-playing is utilized* 

■ 4* role-playing as a component of training as well as evaluation could inplude 

both beginners ^and well trained parents, the latter providing believable, renlis- 

tic models to facilitate the outcome i^f training.^ V j 

Ceietalnly there are limitations in evaluating parents' behavior yla role-playing. 

yPrompts, modelUng; and feedback provided in ;!the role-playing situation t\oe4 to be faicd 
^ , . ■ . . ^ \ ■ . \. . ■ _ V 

out to promote generalization* Role-playlngUocs not 'give us infortnaticfn as to wh-sther. 

Daret>tfl> tti £.ict, do emit the appropriate beh'aviorsjat home. Nevertheless It Is a £or- 



mat vhlch will^hov that bar^itts who emitted a variety of inappropriate protapta aitd con^* 
sequences for behavior can \be traiucd to behave a;)proprint:cly in a c^ir.fc'al .y.oio-pUiy- 
ing situation. Behaving api^ropri^tely urider those conditions may or may not be related ^ 
to effective changes in home behavior. This is an empirical question which will be in* 
^vestl^ated to establish the validity of clinical role-playilig as an inexpensive pro* 
cedure for determining the changes in Jbehavioral targets In parent training situations 
and the generality of training effects over time, - 

Generalization and Multi^level Outcomes 

. ■ ' ' - : \ 

Thi? most meaningful evaluation o^, parent^ training rests on the generalization of 
training effects across behaviors (response generalization) ^ across settings and children 
(stimulus generallzatlpn) f and across time'^^urability) • We hiave collected some data 
which suggest limited generalization of %he effects of the Parent WorC:shop model into 
the home and over time. In Figure 1- Is sliowri. the results of our, follov-up phone and 
home contacts with parents who attended six ofi^our workshops. While oyer 50 percent V 
oi parant?s report concrete*" examples of using t^te behavioral principles learned during - 
the workshop, only a few continue recording behaviors --t continued use of records al- 
ways is associated with home token economies. Iti is generally assumed that the group 

wc^rkshop format is an ecohomi^cal intervention whjLch uses minimal amounts of professional 

\ \ ■■■■ :^ .. i V ■ i ^ - ■ ■.- ■ . ■ . ' . ■ ■ 

time and money to" promote great deal of behavior change. We must rigorously question 



how much behavior of parents al^d children our workshops are actually changing by assess* 



ing generalization in the three i dimensions of response, setting » and tlme« 

Evaluation of Program Outdome $houljd also emphasize multi-level or multiple 
sources of'data,' Cost as >/ell as effective change in behaviors are relevant to outcome, 
particularly to ladtptnistr^tors , granting agencies, politicians , and' taxpayers. Consumer 
satl^^factlon should be ascertained iiS well as behavioral outcomes* If the behavior of 
a child or parent 1^ reliably demonstrate^ to have changed but the parents are unhappy 
and dissatisfied wlt)i the results or the methods, then the trailing procedure may have 
to be altered* The impact of Parent Wbrkshops on the total humm service system also 
will have to be assessed, For instance, does providing Parent t^orkshops reduce deltn- 
O ncy rates > admi^ssions to residential treatment facilities f of children, and drop-out 



tates froTa schools? x . , 

If the workehop format Is to cr.ntinue as a viable approach to child therapy, 
then evaluation issues must^ be comprehensively fon^ulated^ Many who have been, working 
in the field are convinced that the workshops produce' changes; but we are no longer 
interested solely In whether parents can be trained to report' change^ in their children 
behavior • We now want to know the most efficient ways to teach child management, the 
crucial factors in this training proce3s> and how to tiake changes inj parents ^ child ^ 
management skills more durable. The group format for parent training is certainly 
economical in terms of the professional consultant's time. Whether ^t is the most ef- 
' fecttve format to produce the quickest and most durable outcomes can only be determined 
by measurement. Therefore, methods of evaluation are crucial if we are to demonstrate 
and improve on the effectiveness of the parent t«>rkshop approach to <jhild therapy^ We 
have few reasons to believe that our current efforts in parent training could not be' 
improved, ' . - 
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1 . ES^ALUATION OF PROGRAM STRUCTURE 

2. EVALUATION OF PROGRAM PROGRESS 

3. EVALUATJON' OF PROGRAM OUTCOME 

4. COST- EFFECTIVENESS ' 

5. SYSTEMS ANALYSIS 
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